Modifying readable and focusable elements on a page during execution of automated scripts

ABSTRACT

A device may initiate an automated script to perform one or more interactions with a browser application and identify a first element in a page rendered by the browser application that satisfies one or more accessibility criteria, wherein the first element may include text that is readable by a screen reader application and/or an attribute that causes the first element to be navigable using a keyboard. The device may modify the first element to be inaccessible to the screen reader application and the keyboard and insert, into the page, a second element including text that is readable by the screen reader application to describe the one or more interactions that the automated script is performing. The client device may restore the page to an original state based on determining that the automated script has finished executing.

RELATED APPLICATION

This application is a divisional of U.S. patent application Ser. No.16/413,169, filed May 15, 2019, which is incorporated herein byreference.

BACKGROUND

A browser extension or an application may execute a script (e.g., codewritten in a scripting language, such as JavaScript and/or the like) toperform one or more operations in association with a web browser or theapplication. For example, the script can be loaded, injected, orotherwise executed in a context of a particular web page, and the scriptmay access and manipulate elements of the web page. Accordingly, scriptscan be used to improve layout, automate routine or repetitive tasks(e.g., form filling), add new functions, and/or the like.

SUMMARY

According to some implementations, a method may include initiating, by adevice, an automated script to perform one or more interactions with abrowser application executing on the device. The method may includeidentifying, by the device, a set of elements in a page rendered by thebrowser application that satisfy one or more accessibility criteria. Theset of elements may include at least one content element that includestext readable by a screen reader application executing on the device andat least one focusable element that is navigable using a keyboardassociated with the device. The method may include modifying, by thedevice, content on the page to make the text associated with the atleast one content element inaccessible to the screen reader applicationand to make the at least one focusable element inaccessible using thekeyboard associated with the device. The method may include inserting,by the device, at least one explanatory element into the page based onthe one or more interactions that the automated script is performing.The at least one explanatory element may include text that is readableby the screen reader application to describe the one or moreinteractions that the automated script is performing, and the at leastone explanatory element may be navigable using the keyboard associatedwith the device.

According to some implementations, a device may include one or morememories and one or more processors communicatively coupled to the oneor more memories. The one or more processors may be configured toinitiate an automated script to perform one or more interactions with abrowser application executing on the device. The one or more processorsmay be configured to identify a first element in a page rendered by thebrowser application that satisfies one or more accessibility criteria.The first element may include text that is readable by a screen readerapplication executing on the device or an attribute that causes thefirst element to be navigable using a keyboard. The one or moreprocessors may be configured to modify the first element to beinaccessible to the screen reader application and the keyboard. The oneor more processors may be configured to insert, into the page, a secondelement that is readable by the screen reader application based on theone or more interactions that the automated script is performing. Thesecond element may include text to describe the one or more interactionsthat the automated script is performing. The one or more processors maybe configured to restore the page to an original state based ondetermining that the automated script has finished executing.

According to some implementations, a non-transitory computer-readablemedium may store one or more instructions that, when executed by one ormore processors, may cause the one or more processors to initiate ascript to automate one or more interactions with a browser application.The one or more instructions may cause the one or more processors toidentify, in a page rendered by the browser application, a set ofelements that include text readable by a screen reader application. Theone or more instructions may cause the one or more processors to insertat least one explanatory element into the page based on the one or moreinteractions that the script is performing. The at least one explanatoryelement may include text that is readable by the screen readerapplication to describe the one or more interactions that the script isperforming. The one or more instructions may cause the one or moreprocessors to modify a document object model associated with the page tomake the text associated with the set of elements inaccessible to thescreen reader application and to make the text associated with the atleast one explanatory element accessible to the screen readerapplication. The one or more instructions may cause the one or moreprocessors to provide information related to an original state of thedocument object model and one or more modifications to the documentobject model to a device configured to derive one or more accessibilitycriteria for modifying page elements based on the information related tothe original state of the document object model and the one or moremodifications to the document object model.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-1E are diagrams of one or more example implementationsdescribed herein.

FIG. 2 is a diagram of an example environment in which systems and/ormethods described herein may be implemented.

FIG. 3 is a diagram of example components of one or more devices of FIG.2.

FIGS. 4-6 are flow charts of example processes for modifying readableand focusable elements on a page during execution of automated scripts.

DETAILED DESCRIPTION

The following detailed description of example implementations refers tothe accompanying drawings. The same reference numbers in differentdrawings may identify the same or similar elements.

In human-computer interaction, computer accessibility refers to theaccessibility of a computer system to all people, even those withdisabilities or impairments. One area in which accessibility designprinciples are often considered is in context with access to webcontent. Web accessibility design principles (e.g., as described in theWeb Content Accessibility Guidelines (WCAG) published by the WebAccessibility Initiative (WAI) of the World Wide Web Consortium (W3C))are generally intended to ensure that there are no barriers that preventinteraction with or access to web content for people with disabilities(e.g., visual impairments, motor or dexterity impairments, hearingimpairments, cognitive impairments, and/or the like). For example,coding a web page with semantically meaningful content (e.g., textualequivalents for images, meaningfully named links, and/or the like) mayhelp visually impaired users access the content using a screen reader(e.g., text-to-speech software, text-to-Braille hardware, and/or thelike). In another example, when links are underlined or otherwisedifferentiated from other text on the web page, the links may be morenoticeable to visually impaired or color-blind users. In anotherexample, web pages may be coded in a way to enable navigation using akeyboard alone to help users who cannot use a mouse or a standardkeyboard due to motor impairments.

Another technique that can be used to improve web accessibility isthrough browser automation. In particular, a browser applicationexecuting on a client device can be configured to execute one or moreautomated scripts (sometimes referred to as content scripts, userscripts, and/or the like) to perform one or more interactions with thebrowser application and/or a page that is loaded in the browserapplication. For example, the interactions performed by the automatedscript may include input interactions (e.g., keystrokes, mousemovements, button clicks, menu selections, hover states, formsubmissions, and/or the like), opening and/or closing browser windowsand/or browser tabs, navigating pages, and/or the like. In this way, theautomated script can mimic human-computer interactions to carry outcertain workflows without user intervention, which can improve access toweb content for users who may have difficulty seeing, hearing,understanding, and/or using input devices to interact with web content.

Although browser automation can be used to assist users with performingrepetitive or repeatable interactions (or sequences of interactions) andthereby improve accessibility, browser automation tools also tend tointroduce certain accessibility challenges. For example, an automatedscript may introduce accessibility challenges that relate to navigation(e.g., an inability or difficulty to navigate web pages using a keyboardor assistive technology), hidden content (e.g., injecting or presentingcontent or functionality that is not visible and/or accessible toassistive technologies such as a screen reader), an inability ordifficulty for a user to control automated content changes, confusionand/or disorientation (e.g., altering or disabling normal browserfunctionality and/or triggering actions that the user may be unawareof), and/or the like.

Accordingly, while browser automation can be employed to automaticallyperform certain actions, interactions, and/or the like that the userwould otherwise have to perform, which may benefit users with certainimpairments, the automation can inadvertently create confusion regardingwhat is happening, especially for users using accessibility tools. Forexample, a screen reader may read various tags or other elements on thepage that are not relevant to the action(s) that the automated script isperforming, elements or other information that the automated scriptinjects into the page, and/or the like. In some cases, where the user isdisoriented, confused, or otherwise unaware of what is happening in thebrowser, the user may have a poor experience that may fail to complywith accessibility design guidelines (e.g., to present information anduser interface components in a manner that users can perceive, to ensurethat user interface components and navigation are operable and/orunderstandable, to format content in a manner that can be reliablyinterpreted by various user agents, including assistive technologies).

In some cases, the confusion and/or disorientation that a userexperiences while an automated script is running may result insubstantial wasted computing resources (e.g., processor resources,memory resources, communication resources, and/or the like). Forexample, the user may attempt to perform certain actions while theautomated script is running, which may cause the automated script to beunsuccessful and thereby waste computing resources by having to rollback changes that the automated script applied, attempt to execute theautomated script again, and/or the like. In another example, userintervention that causes an automated script to fail may also lead theuser to incorrectly assume that certain actions were successfullyperformed, which can lead to wasted resources where subsequent useractions are based on a false understanding of the state of a browsingsession (e.g., attempting to access an account history even though anautomated script was unable to successfully log into the account).Furthermore, the confusion and/or disorientation could escalate to apoint where the user starts a communication session (e.g., a telephonecall, an online chat, and/or the like) with customer service personnelto ask for help, which can also consume device resources, networkresources, and/or the like. For example, customer service personnel maybe unable to reassure users that automated interactions were successful,which may lead to wasted computing resources (e.g., helping the user toinvestigate whether the script completed the intended workflow). Inother cases, customer service personnel may have no knowledge about theautomated script, which can lead to the communication session betweenthe user and the customer service personnel wasting computing resourceswithout any resolution to the underlying issue that caused the confusionand/or disorientation.

Some implementations described herein may automatically modify contenton a page loaded in a browser application according to one or moreaccessibility criteria while an automated script is executing to performone or more automated interactions with the browser application. Forexample, a browser extension, a plug-in, or another applicationproviding a runtime environment for the automated script may identify aset of elements in the page that satisfy the one or more accessibilitycriteria (e.g., elements that provide text alternatives for images orother non-text content, elements that make functionality triggered bymouse or gesture available via keyboard, and/or the like), and theidentified set of elements may be temporarily modified to reduceconfusion to users that may be using assistive technology such as ascreen reader, a keyboard to navigate the page, and/or the like. Forexample, elements that provide text alternatives, elements that enablekeyboard navigation, and/or the like may be temporarily removed, andadditional elements that provide text to explain the interactions thatthe automated script is performing may be temporarily inserted,emphasized, and/or the like.

In this way, the content on the page may be modified while the automatedscript is executing, to remove or decrease a focus on elements that maynot be relevant to the interactions that the automated script isperforming, and to insert or increase a focus on elements that mayexplain the interactions that the automated script is performing. Inthis way, the user may be less likely to be confused or disorientedwhile the automated script is executing, which helps to ensure that theexecution of the automated script complies with various accessibilityguidelines (e.g., to ensure that information provided by a script isidentified with functional text that can be accessed and read usingassistive technology when the script displays content, creates userinterface elements, performs automated interactions, and/or the like).This also conserves computing resources that would otherwise be wastedor consumed by the user intervening during script execution andpotentially causing the script to fail, attempting to execute the scriptmultiple times, contacting customer service personnel, and/or the like.

FIGS. 1A-1E are diagrams of one or more example implementations 100described herein. As will be described in further detail herein, exampleimplementation(s) 100 may include a client device that may execute abrowser application and one or more automated scripts to automate one ormore interactions with the browser application, a web server that mayhost or otherwise serve web pages to the browser application executingon the client device, and a web accessibility platform that maycommunicate with the client device to provide and/or obtain informationthat relates to one or more accessibility criteria for modifying contentof the web pages displayed in the browser application while the clientdevice is executing the one or more automated scripts to automate theone or more interactions with the browser application.

As shown in FIG. 1A, the client device may modify content associatedwith a page displayed in a browser application while an automated scriptis executing on the client device based on one or more accessibilitycriteria that are configured to emphasize and/or explain functionalityof the automated script and to decrease a focus and/or disable access toother elements that may not be relevant to the functionality of theautomated script. For example, as shown in FIG. 1B, a page may includeone or more content elements that contain text that is readable by ascreen reader application and/or focusable elements that are navigableby a keyboard. As shown in FIG. 1C, the content elements and/orfocusable elements may be temporarily removed from the page ortemporarily made inaccessible (e.g., to assistive technology tools suchas the screen reader application, the keyboard, and/or the like) whilethe automated script is executing. As further shown in FIG. 1C, one ormore readable and/or focusable elements may be inserted into the page toemphasize and/or explain the functionality of the automated script. Asshown in FIG. 1D, the client device may discard original elements on thepage based on determining that the automated script has loaded a newpage. Additionally, or alternatively, as shown in FIG. 1E, the clientdevice may restore the original elements on the page based ondetermining that the automated script has completed executing, and theclient device may provide information related to the original pageelements and/or modifications to the page to the web accessibilityplatform. Accordingly, the web accessibility platform may use theinformation related to the original page elements and/or themodifications to the page to update the one or more accessibilitycriteria used to modify page content while one or more automated scriptsare executing.

As shown in FIG. 1A, and by reference number 105, the client device maycommunicate with a web server to load content associated with a webpage. In particular, as mentioned above, the client device may beexecuting a browser application, and the browser application may requestcontent associated with a particular web page from the web server. Forexample, a user of the client device may enter a Uniform ResourceLocator (URL) into the web browser, which may obtain a web pagecorresponding to the URL from the web server by submitting a HypertextTransfer Protocol (HTTP) request to the web server. The web server mayprovide, to the client device, a response to the HTTP request thatincludes content of the web page, status information relating to theHTTP request, and/or the like. Additionally, or alternatively, the HTTPrequest to obtain the web page may be initiated based on the userclicking a link in another web page, an email, a document, and/or thelike. Once the content of the web page has been obtained, the browserapplication may render the content of the web page on the client device(e.g., in a particular tab where the browser application supports atabbed document interface that allows multiple web pages to be containedin a single browser window).

As further shown in FIG. 1A, and by reference number 110, the clientdevice may obtain one or more accessibility criteria for modifyingcontent of pages rendered by the browser application during execution ofautomated scripts. For example, one web accessibility design principlemay be to ensure that information and user interface components arepresentable to users with impairments in a way that the users canperceive. Accordingly, web pages often include one or more elements thatmay enable visually impaired users to receive information via sound(e.g., using a screen reader application) and/or touch (e.g., usingtext-to-Braille hardware). For example, in Hypertext Markup Language(HTML), an “alt” attribute can be used to provide a text alternative fornon-text content (e.g., images, buttons, icons, graphics, datarepresented on charts or diagrams, form controls, user interfacecomponents, and/or the like). In general, a text alternative may bereadable by a screen reader application, which may enable visuallyimpaired users to understand and/or interact with the element(s)associated with the text alternative.

However, while an automated script is executing, text accessible to thescreen reader application may not be relevant to the functionality thatthe automated script is performing, which could lead to user confusion,disorientation, distraction, and/or the like if the screen readerapplication were to read all of the text accessible to the screen readerapplication. Accordingly, in some implementations, the one or moreaccessibility criteria may be used to identify content elements thatinclude text that is accessible (e.g., readable) by a screen readerapplication, which may enable temporary modification of the contentelements that include the text accessible to the screen readerapplication while the automated script is executing. For example, theone or more accessibility criteria may define elements, attributes,and/or the like that typically include or are otherwise associated withreadable text (e.g., an alt attribute, regular text in an elementsection between an opening tag and a closing tag, a page title, asemantic header to assist in navigation, a link title, and/or the like).In some implementations, the content elements that include textaccessible by the screen reader application may be temporarily modified(e.g., while an automated script is executing) to make the textinaccessible by the screen reader application.

In some implementations, another web accessibility design principle maybe to ensure that functions that can be triggered via a mouseinteraction or a gesture interaction can also be operated using akeyboard. For example, some users (e.g., with motor impairments, visualimpairments, and/or the like) may not use a mouse and instead rely on akeyboard and/or hardware that mimics keyboard functionality to interactwith web content. Accordingly, web pages often include one or moreelements that enable keyboard navigation to access functionality such asform controls, input boxes, user interface components, and/or the like.In this way, the elements that enable keyboard navigation may improveaccessibility for keyboard users (including people using alternativekeyboards such as keyboards with ergonomic layouts, on-screen keyboards,switch devices, and/or the like). Furthermore, the elements that enablekeyboard access may assist people using voice recognition or speechinputs to operate websites and to dictate text through a keyboardinterface. Further still, text alternatives can be used in combinationwith elements that enable keyboard access to provide contextual labelsfor controls and functionality in order to aid navigation via keyboard,voice, and/or the like.

In some implementations, the elements that enable keyboard navigationmay include one or more “tabindex” attributes, which are generally usedto indicate whether an element can be focused, if and/or where theelement is located in a sequence of elements navigable using thekeyboard (e.g., using a “Tab” key), and/or the like. For example, thetabindex attribute is generally associated with an integer value, wherean element having a tabindex attribute equal to zero (0) is focusable insequential keyboard navigation in a default focus order (e.g.,determined according to a position of the element in the source HTML).Where the element has a positive value for the tabindex attribute, theelement is focusable in sequential keyboard navigation and a position ofthe element in the sequence of elements navigable using the keyboard isdetermined based on the value for the tabindex attribute (e.g., anelement having a tabindex equal to one (1) will be in focus before anelement having a tabindex equal to two (2)). However, where the elementhas a negative value for the tabindex attribute, the element may befocusable but not accessible by sequential keyboard navigation. In otherwords, an element with a negative tabindex attribute may be omitted fromthe sequence of elements navigable using the keyboard.

Additionally, or alternatively, the elements that enable keyboardnavigation may include interactive elements that are natively navigableusing a keyboard (e.g., without a tabindex attribute being required inorder to give the element keyboard focus). For example, in HTML,interactive elements that are natively navigable using a keyboard mayinclude links, buttons, form controls, and/or the like. Additionally, oralternatively, the elements that enable keyboard navigation may includeone or more “accesskey” attributes used to assign keyboard shortcuts tothe elements that include the accesskey attribute(s). For example, theaccesskey attribute can be included within a particular element todefine a keyboard shortcut that will navigate directly to the particularelement (e.g., an element including attribute accesskey=“s” may beaccessed by a keyboard shortcut in which a user presses “s” incombination with one or more activation keys, such as Alt, Shift+Alt,Control+Option, and/or the like).

However, while an automated script is executing, one or more elementsthat are focusable, navigable, or otherwise accessible using a keyboardmay not be relevant to the functionality that the automated script isperforming, which could lead to user confusion, disorientation,distraction, and/or the like. For example, if the client device isexecuting a screen reader application, the screen reader applicationwould read text associated with each element that the user accesses (or“tabs” to) using the keyboard, and the text may be irrelevant to thefunctionality that the automated script is performing. Additionally, oralternatively, one or more of the elements that are accessible using thekeyboard may be interactive (e.g., buttons, hyperlinks, and/or thelike), and allowing keyboard access to an interactive element may resultin the user selecting or otherwise interacting with the element in amanner that interferes with the functionality that the automated scriptis performing. For example, the automated script may be configured toautomatically navigate a sequence of pages to swap a credit card thatthe user has on-file at a particular website, and allowing the user totab to an interactive element and select the interactive element maycause the browser to navigate away from the sequence of pages used toswap the user's credit card. Accordingly, in some implementations, theone or more accessibility criteria may be used to identify focusableelements that are navigable using a keyboard, which may be temporarilymodified to be made inaccessible to the keyboard while the automatedscript is executing.

In some implementations, the accessibility criteria for modifying pagecontent may be used to identify other elements, attributes, and/or thelike that may cause user confusion, user disorientation, userdistraction, functional interference, and/or the like while an automatedscript is executing. For example, in addition to providing criteria foridentifying text content that is accessible to a screen readerapplication and/or focusable elements that the user may traverse using akeyboard, the accessibility criteria may include one or more indicatorsto handle interactive (e.g., clickable) elements, advertising content,multimedia content, and/or the like. For example, the accessibilitycriteria may include one or more indicators for identifying interactiveelements such as links, buttons, form controls, and/or the like, and atype of tag associated with the interactive elements may be modified tomake the interactive elements hidden from the screen reader application,inaccessible using the keyboard, and unselectable (e.g., a button may betemporarily changed to a different user interface component that doesnot appear clickable, a link may be temporarily changed to anon-interactive element such as a span element, and/or the like). Inanother example, multimedia content (e.g., advertising content, embeddedvideo and/or audio content, and/or the like) often has visual and/orauditory characteristics that are not designed with screen readers orother assistive technology in mind. Accordingly, in someimplementations, the accessibility criteria may include one or moreindicators for detecting multimedia content (e.g., based on an inlineframe or “iframe” that represents a nested browsing context embedded ina current web page, an identifier of a third-party from which thecontent originates, and/or the like), which may be modified to beinaccessible to the screen reader application, to be inaccessible viakeyboard navigation, to be playable only by selection of a play button(e.g., with automatic play disabled), and/or the like.

As further shown in FIG. 1A, and by reference number 115, the clientdevice may be configured to modify content on the page loaded from theweb server based on the accessibility criteria obtained from the webaccessibility platform. In particular, the client device may beexecuting a browser extension, a browser plug-in, and/or anothersuitable application that can be used to initiate an automated script toautomatically perform one or more interactions with a browserapplication in which the page is loaded, and the content on the page maybe modified while the automated script is executing to perform the oneor more interactions. For example, in some implementations, theautomated script can include a content script, a user script, aforeground script, and/or the like, which may execute in a context of aparticular page and have permission to read and modify content of theparticular page using one or more document object model (DOM)application program interfaces.

Furthermore, in some implementations, the automated script can beassociated with a background script used to maintain long-term stateand/or perform long-term operations independently of a lifetime of anyparticular web page or browser window. Accordingly, in someimplementations, the client device may load and execute the backgroundscript when the browser extension, browser plug-in, and/or othersuitable application is loaded, and the background script may be used toinitiate and manage one or more automated scripts that execute in acontext of particular pages (e.g., browser tabs). In someimplementations, when the background script initiates the one or moreautomated scripts, the background script may modify the content on thepage based on the accessibility criteria, instruct the foreground scriptto modify the content on the page based on the accessibility criteria,initiate a separate process to modify the content on the page based onthe accessibility criteria, and/or the like.

In some implementations, the client device may obtain the automatedscript(s) from a repository associated with the web accessibilityplatform, a third-party repository, and/or the like. Additionally, oralternatively, the automated script(s) may be created locally at theclient device (e.g., using a script editor that allows users to writecustom script code, using a software utility that can record andsubsequently replay user interactions such as button clicks, menuselections, hover states, form submissions, and/or the like). In someimplementations, the automated script(s) may be initiated manually(e.g., based on a user command) or automatically based on a triggerevent (e.g., when the page loaded from the web server has a URL thatmatches a given pattern, when a particular browser action is initiated,and/or the like).

In some implementations, the automated script(s) may be configured toperform one or more interactions at a graphical interface level in orderto carry out one or more interactions that would normally be performedby a user (e.g., selecting links, selecting from a menu, typing into atext box, navigating through pages, extracting data from pages, and/orthe like). In this case, the interactions performed by the automatedscript(s) may be visually perceptible, but the interactions maynonetheless cause confusion, disorientation, and/or the like because theuser may not know what actions are being performed by the automatedscript(s). The potential confusion, disorientation, and/or the like maybe worse in cases where the user is visually impaired, usingaccessibility tools, and/or the like because the user may not be able toperceive the actions that are being performed by the automatedscript(s).

Additionally, or alternatively, the automated script(s) may inject oneor more inline frames (e.g., iframe elements) into a current page tocreate a nested browsing context embedded in the current page, and theautomated script(s) can be executed within the nested browsing contextto remain hidden from sight. However, if the actions being performed bythe automated script(s) are hidden from sight, similar issues may arisein that the user may be unable to perceive the actions that are beingperformed by the automated script(s). Furthermore, depending on wherethe nested browsing context is placed within the page and/or a screen ofthe client device, accessibility tools such as screen readerapplications may have difficulty finding the information injected intothe nested browsing context.

Accordingly, in some implementations, the page content may be modifiedto focus on or otherwise emphasize the actions that are being performedby the automated script(s). For example, the page may be associated witha document object model (DOM) that represents the page using a treestructure. Each branch in the tree may end in a node, and each node maycontain one or more objects. In some implementations, variousprogrammatic methods (e.g., application program interfaces) can be usedto access and manipulate elements of the DOM in order to change astructure, style, content, and/or other characteristics of the page.Accordingly, when an automated script is initiated to perform one ormore interactions with a browser application (e.g., a particular pageloaded in the browser application), the DOM associated with a page thatthe automated script is interacting with may be parsed to identify oneor more elements, attributes, and/or the like that may cause userconfusion, disorientation, and/or the like.

For example, as mentioned above, the DOM may be parsed to identifycontent elements containing text that is readable to a screen readerapplication and to identify focusable elements that are navigable usinga keyboard. The content elements may be modified to be inaccessible tothe screen reader application, and the focusable elements may bemodified to be inaccessible using a keyboard. Additionally, oralternatively, multimedia elements may be disabled, or automatic playfor multimedia elements may be disabled, and interactive elements may bemade unselectable to prevent the user from initiating actions that mayinterfere with the actions being performed by the automated script.Furthermore, in some implementations, one or more explanatory elementsmay be inserted into the DOM to explain the actions being performed bythe automated script. For example, the explanatory element(s) mayinclude text to describe the actions being performed by the automatedscript, and the explanatory element(s) may further include one or moreattributes that make the explanatory element(s) navigable using akeyboard.

In this way, by removing text and/or focus from existing elements on thepage and inserting text and/or focus for explanatory elements thatrelate to the actions being performed by the automated script, the pagecontent may be modified to focus on or otherwise emphasize the actionsthat are being performed by the automated script(s). For example, theuser may press a tab key to navigate directly to the explanatoryelements that relate to the actions being performed by the automatedscript, and the screen reader may read the text to describe the actionsthat the automated script is performing. In this way, the user may beinformed about the actions being performed by the automated scriptwithout being distracted or confused by other page elements. Forexample, the DOM may be parsed to identify any elements that includetext that is accessible to the screen reader (e.g., regular text on thepage, text associated with an alt attribute, and/or the like), andoriginal text may be removed from the page such that the screen readerapplication has nothing to read except for the explanatory elements thatare inserted to explain the actions that the automated script isperforming. Furthermore, any elements that are navigable using akeyboard may be associated with a tabindex attribute having a negativevalue to make the elements inaccessible using a keyboard, and theexplanatory element may be associated with a tabindex attribute having anon-negative value to ensure that the explanatory element will benavigable using the keyboard. In some implementations, the non-negativevalue may be set to one (1) to ensure that the explanatory element willbe first in a sequence of elements that are navigable using thekeyboard.

In some implementations, when modifying the page content, an originalstate of one or more elements that are modified may be preserved, toenable restoration of the page content when the automated script hasfinished executing. For example, an element may be associated with an“id” attribute that specifies a unique identifier for the element withina context of the page (i.e., the id attribute is unique within thepage). Accordingly, for an element that includes an id attribute, anoriginal state of the element may be stored (e.g., in a cache associatedwith the browser application, a memory of the client device, the webaccessibility platform, and/or the like) in association with the idattribute. Additionally, or alternatively, for an element that does notinclude an id attribute, a unique id attribute may be generated for theelement and the original state of the element may be stored inassociation with the generated id attribute. In this way, an originalstate of the content on the page (e.g., prior to modification) may becached, stored, or otherwise preserved, which may allow the originalstate of the content on the page to be restored when the automatedscript has finished executing.

In some implementations, multiple automated scripts may be executedsequentially and/or in parallel, or an automated script may execute onmultiple pages (e.g., where the automated script is configured tonavigate among different pages to carry out an intended function). Insuch cases, when the automated script loads a new page, the originalstate of the page that was modified may be discarded because the page isno longer being rendered by the browser application, and a backgroundscript may cause the automated script to perform the process ofmodifying content on the new page in a similar manner as describedabove.

Example implementation(s) 100 will now be described with reference toFIGS. 1B-1C, which illustrate example modifications to HTML code thatmay be applied while an automated script is executing to emphasizeand/or explain functionality of the automated script and to decrease afocus and/or disable access to other elements that may not be relevantto the functionality of the automated script. For example, as shown inFIG. 1B, and by reference number 120, an HTML page includes an image(img) element that has a source attribute to specify an address (e.g., aURL) of the image. The image element also includes an alt attribute thatis accessible to a screen reader application, and which may be displayedin place of the image if the image file is not loaded (e.g., because theuser has chosen to disable images, the image is unable to be loaded fromthe specified address, and/or the like). For example, as illustrated inFIG. 1B, the alt attribute includes the text “A screen reader will readthis text, which describes the content and function of the image.” TheHTML page shown in FIG. 1B also includes a paragraph element, whichincludes the text “The screen reader will also read this,” and a pair ofbuttons that include readable text and tabindex attributes that define asequence in which the buttons are navigable using a keyboard.Accordingly, as further shown in FIG. 1B, and by reference number 125,the HTML page may be rendered in a web browser. In particular, the imagefile is displayed in the web browser along with the text in theparagraph element, and the pair of buttons are further displayed in theweb browser along with the readable text.

As shown in FIG. 1C, and by reference number 130, the content of theHTML page is modified when an automated script is executed to makeoriginal text inaccessible to the screen reader application, to makeoriginal focusable elements inaccessible using a keyboard, and to insertan explanatory element that describes the actions that the automatedscript is performing. For example, the text contained in the altattribute of the image element has been removed (note that the altattribute is empty to comply with standards requiring that each imagehave an alt attribute), the text from the subsequent paragraph elementhas been removed, and the text displayed within the buttons has beenremoved. Furthermore, each of these original elements has beenassociated with a tabindex having a negative value, which causes thecorresponding element to be omitted from a sequence of elements that arenavigable using a keyboard. Additionally, as shown by reference number130, each original element that has been modified is associated with anid attribute that is unique within the HTML page. Accordingly, anoriginal state of each element (e.g., as shown in FIG. 1B) may be cachedor otherwise stored in association with the corresponding id attribute,which may enable the original state of each element to be subsequentlyrestored when the automated script finishes executing.

In some implementations, in addition to modifying original elements tomake original text inaccessible to the screen reader application and tomake original focusable elements inaccessible using a keyboard, the HTMLpage has been modified to include an explanatory element that describesthe actions being performed by the automated script. In particular, asshown by reference number 130, a button element has been inserted intothe page, and the button element includes the text “We are performingyour automated operation now. Please wait.” Accordingly, when a screenreader application processes the page as shown by reference number 135,the only text that remains accessible to the screen reader applicationis the text associated with the explanatory element. Furthermore, asshown by reference number 130, the explanatory element includes atabindex having a value of one (1), which ensures that the explanatoryelement will be first in a sequence of elements that are navigable usinga keyboard. Accordingly, if a user presses “tab” to navigate through therendered page, focus will be initially directed at the explanatoryelement. In this way, if the user is using a keyboard to navigate thepage, the only element to which the keyboard will navigate is theexplanatory element. In this way, the user is not required to tabthrough the entire page looking for the explanatory element or to havethe screen reader application read other elements that are not relevantto the actions being performed by the automated script.

As shown in FIG. 1D, and by reference number 140, the client device maystore an original state of page elements that are modified (e.g., in acache associated with the web browser, a memory of the client device,and/or the like) to enable subsequent restoration of the original stateof the modified page elements. For example, in context with FIGS. 1B-1C,the client device may store an original state of the image element, theparagraph element, and the pair of button elements in association withrespective identifier (id) attributes, which are inserted into themodified page elements. In some cases, one or more elements mayoriginally include an identifier attribute, and an identifier attributemay be generated for any modified elements that do not have an existingidentifier attribute. In this way, the original state of the page may bepreserved, which may enable the original state of the page to berestored when the automated script has finished executing. However, incases where the automated script navigates between different pages toperform a sequence of interactions across the different pages, theoriginal state that is stored in cache, memory, and/or the like can beremoved to conserve memory resources.

Accordingly, as shown in FIG. 1D, and by reference number 145, theautomated script may communicate with a web server to load a new page.For example, the automated script may select a link in the previouspage, submit a form via the previous page, and/or the like. As furthershown in FIG. 1D, and by reference number 150, the original pageelements may be discarded based on the automated script loading the newpage. In this way, memory resources are conserved because the automatedscript has navigated away from the previous page associated with theoriginal page elements that are stored in cache, memory, and/or thelike.

As shown in FIG. 1E, and by reference number 155, the client device maydetect that the automated script has completed executing. For example,the client device may execute a background script to maintain long-termstate and/or perform long-term operations independently of a lifetime ofany particular web page or browser window, and the background script maybe configured to use one or more message passing application programinterfaces to communicate with one or more automated content scripts,user scripts, foreground scripts, and/or the like that have directaccess to web pages rendered in a particular browser window, browsertab, and/or the like. Accordingly, when the automated script hascompleted, the automated script may notify the background script via themessage passing application program interface.

As further shown in FIG. 1E, and by reference number 160, the clientdevice may cause an original state of content on the page to be restoredbased on the automated script completing. For example, the client devicemay retrieve an original page element from storage, determine anidentifier attribute that is associated with the original page element,and use the original page element to replace an element in the modifiedpage that includes the identifier attribute. This process may berepeated for each original page element that is preserved in storage,thus restoring the original page content.

As further shown in FIG. 1E, and by reference number 165, the clientdevice may provide information related to the original page elements andinformation related to a set of modifications that were applied to theoriginal page elements to the web accessibility platform. For example,the client device may provide the web accessibility platform with anoriginal document object model (DOM) prior to applying anymodifications, a modified version of the DOM after the modifications areapplied, and/or the like. Additionally, or alternatively, the clientdevice may provide information related to individual page elements thatwere modified. As further shown in FIG. 1E, and by reference number 170,the web accessibility platform may analyze the information related tothe original page elements and the information related to a set ofmodifications that were applied to the original page elements to updatethe criteria for removing content from and/or inserting content into webpages while automated scripts are executing. For example, the webaccessibility platform may employ one or more accessibility testingtools that can evaluate page content for accessibility problems,compliance with accessibility guidelines, and/or the like. In this way,the web accessibility platform may detect certain elements, attributes,coding styles, patterns, and/or the like that may potentially cause auser to be confused, disoriented, distracted, and/or the like while anautomated script is executing, which may be used to update the criteriafor removing and/or inserting page content to reduce a probability ofsuch confusion, disorientation, distraction, and/or the like.

As indicated above, FIGS. 1A-1E are provided as one or more examples.Other examples may differ from what is described with regard to FIGS.1A-1E.

FIG. 2 is a diagram of an example environment 200 in which systemsand/or methods described herein may be implemented. As shown in FIG. 2,environment 200 may include one or more client devices 210 that mayexecute one or more applications (e.g., a web browser 212, a browserextension 214, a screen reader 216, and/or the like), a web server 220,a web accessibility platform 230 in a cloud computing environment 240that includes a set of computing resources 245, and a network 250.Devices of environment 200 may interconnect via wired connections,wireless connections, or a combination of wired and wirelessconnections.

Client device 210 includes a device that supports web browsing and/ornetwork access via an application. For example, client device 210 mayinclude a computer (e.g., a desktop computer, a laptop computer, atablet computer, a handheld computer, and/or the like), a mobile phone(e.g., a smart phone and/or the like), a television (e.g., a smarttelevision), an interactive display screen, and/or a similar type ofdevice. Client device 210 may host a web browser 212 and/or a browserextension 214 installed on and/or executing on the client device 210.Client device 210 may further host a screen reader 216 that can readinformation on a screen of client device 210, including contentassociated with one or more pages that are rendered by the web browser212. Additionally, or alternatively, client device 210 may host one ormore applications that permit access to a network (e.g., network 250),which may enable client device 210 to load web pages from web server220, obtain automated scripts and/or accessibility criteria formodifying web pages loaded from web server 220 from web accessibilityplatform 230, and/or the like.

Web browser 212 includes an application, executing on client device 210,that supports web browsing. For example, web browser 212 may be used toaccess information on the World Wide Web, such as web pages, images,videos, and/or other web resources. Web browser 212 may access such webresources using a uniform resource identifier (URI), such as a uniformresource locator (URL), a uniform resource name (URN), and/or the like.Web browser 212 may enable client device 210 to retrieve and present,for display, content of a web page.

Browser extension 214 includes an application, executing on clientdevice 210, capable of extending or enhancing functionality of webbrowser 212. For example, browser extension 214 may be a plug-inapplication for web browser 212. Browser extension 214 may be capable ofexecuting one or more automated scripts (e.g., code, which may bewritten in a scripting language, such as JavaScript and/or the like) toperform an operation in association with web browser 212. For example,the automated script(s) may be initiated to automatically perform one ormore interactions with web browser 212 (e.g., navigating pages,selecting links, inputting keystrokes, mimicking mouse movements, and/orthe like).

Screen reader 216 includes an application, executing on client device210, capable of converting digital text into synthesized speech.Additionally, or alternatively, screen reader 216 may interact withtext-to-Braille hardware and/or other non-visual output modalities toallow the digital text to be perceived by a user who is blind, visuallyimpaired, illiterate, cognitively impaired, and/or the like.

Web server 220 includes a device capable of serving web content (e.g.,web documents, HTML documents, web resources, images, style sheets,scripts, text, and/or the like). For example, web server 220 may includea server and/or computing resources of a server, which may be includedin a data center, a cloud computing environment, and/or the like. Webserver 220 may process incoming network requests (e.g., from clientdevice 210) using HTTP and/or another protocol. Web server 220 maystore, process, and/or deliver web pages to client device 210. In someimplementations, communication between web server 220 and client device210 may take place using HTTP.

Web accessibility platform 230 includes one or more devices capable ofreceiving, generating, storing, processing, and/or providing informationassociated with modifying web page content rendered by web browser 212while browser extension 214 is executing one or more automated scripts.For example, as described elsewhere herein, web accessibility platform230 may provide client device 210 with one or more accessibilitycriteria for identifying elements, attributes, and/or the like in a pagethat are to be modified while an automated script is executing (e.g.,elements, attributes, and/or the like that contain text accessible byscreen reader 216, elements that are navigable using a keyboard, and/orthe like). In some implementations, web accessibility platform 230 mayreceive information from client device 210 that relates to one or morepages, elements, attributes, and/or the like that are modified while anautomated script is executing, and web accessibility platform 230 mayanalyze the information received from client device 210 to update theaccessibility criteria to reduce user confusion, disorientation,distraction, and/or the like that may occur while an automated script isexecuting.

In some implementations, as shown, web accessibility platform 230 can behosted in a cloud computing environment 240. Notably, whileimplementations described herein describe web accessibility platform 230as being hosted in cloud computing environment 240, in someimplementations, web accessibility platform 230 can be non-cloud-based(e.g., can be implemented outside of a cloud computing environment) orpartially cloud-based.

Cloud computing environment 240 includes an environment that hosts webaccessibility platform 230. Cloud computing environment 240 can providecomputation services, software services, data access services, storageservices, and/or other services that do not require end-user knowledgeof a physical location and configuration of system(s) and/or device(s)that host web accessibility platform 230. As shown, cloud computingenvironment 240 can include a set of one or more computing resources 245(referred to collectively as “computing resources 245” and individuallyas “computing resource 245”).

Computing resource 245 includes one or more personal computers,workstation computers, server devices, or other types of computationand/or communication devices. In some implementations, computingresource 245 can host web accessibility platform 230. The cloudresources can include compute instances executing in computing resource245, storage devices provided in computing resource 245, data transferdevices provided by computing resource 245, and/or the like. In someimplementations, computing resource 245 can communicate with othercomputing resources 245 via wired connections, wireless connections, ora combination of wired and wireless connections.

As further shown in FIG. 2, computing resource 245 includes a group ofcloud resources, such as one or more applications (“APPs”) 245-1, one ormore virtual machines (“VMs”) 245-2, virtualized storage (“VSs”) 245-3,one or more hypervisors (“HYPs”) 245-4, and/or the like.

Application 245-1 includes one or more software applications that can beprovided to or accessed by client device 210, web server 220, and/or thelike. Application 245-1 can eliminate a need to install and execute thesoftware applications on client device 210, web server 220, and/or thelike. For example, application 245-1 can include software associatedwith web accessibility platform 230 and/or any other software capable ofbeing provided via cloud computing environment 240. In someimplementations, one application 245-1 can send information to and/orreceive information from one or more other applications 245-1, viavirtual machine 245-2.

Virtual machine 245-2 includes a software implementation of a machine(e.g., a computer) that executes programs like a physical machine.Virtual machine 245-2 can be either a system virtual machine or aprocess virtual machine, depending upon use and degree of correspondenceto any real machine by virtual machine 245-2. A system virtual machinecan provide a complete system platform that supports execution of acomplete operating system (“OS”). A process virtual machine can executea single program, and can support a single process. In someimplementations, virtual machine 245-2 can execute on behalf of a user(e.g., a user of client device 210, web server 220, and/or the like),and can manage infrastructure of cloud computing environment 240, suchas data management, synchronization, or long-duration data transfers.

Virtualized storage 245-3 includes one or more storage systems and/orone or more devices that use virtualization techniques within thestorage systems or devices of computing resource 245. In someimplementations, within the context of a storage system, types ofvirtualizations can include block virtualization and filevirtualization. Block virtualization can refer to abstraction (orseparation) of logical storage from physical storage so that the storagesystem can be accessed without regard to physical storage orheterogeneous structure. The separation can provide administrators ofthe storage system with flexibility in how the administrators managestorage for end users. File virtualization can eliminate dependenciesbetween data accessed at a file level and a location where files arephysically stored. This can enable optimization of storage use, serverconsolidation, and/or performance of non-disruptive file migrations.

Hypervisor 245-4 can provide hardware virtualization techniques thatallow multiple operating systems (e.g., “guest operating systems”) toexecute concurrently on a host computer, such as computing resource 245.Hypervisor 245-4 can present a virtual operating platform to the guestoperating systems, and can manage the execution of the guest operatingsystems. Multiple instances of a variety of operating systems can sharevirtualized hardware resources.

Network 250 includes one or more wired and/or wireless networks. Forexample, network 250 can include a cellular network (e.g., a long-termevolution (LTE) network, a code division multiple access (CDMA) network,a 3G network, a 4G network, a 5G network, another type of nextgeneration network, and/or the like), a public land mobile network(PLMN), a local area network (LAN), a wide area network (WAN), ametropolitan area network (MAN), a telephone network (e.g., the PublicSwitched Telephone Network (PSTN)), a private network, an ad hocnetwork, an intranet, the Internet, a fiber optic-based network, a cloudcomputing network, a core network, and/or the like, and/or a combinationof these or other networks.

The number and arrangement of devices and networks shown in FIG. 2 areprovided as one or more examples. In practice, there may be additionaldevices and/or networks, fewer devices and/or networks, differentdevices and/or networks, or differently arranged devices and/or networksthan those shown in FIG. 2. Furthermore, two or more devices shown inFIG. 2 may be implemented within a single device, or a single deviceshown in FIG. 2 may be implemented as multiple, distributed devices.Additionally, or alternatively, a set of devices (e.g., one or moredevices) of environment 200 may perform one or more functions describedas being performed by another set of devices of environment 200.

FIG. 3 is a diagram of example components of a device 300. Device 300may correspond to client device 210, web server 220, and/or webaccessibility platform 230. In some implementations, client device 210,web server 220, and/or web accessibility platform 230 may include one ormore devices 300 and/or one or more components of device 300. As shownin FIG. 3, device 300 may include a bus 310, a processor 320, a memory330, a storage component 340, an input component 350, an outputcomponent 360, and a communication interface 370.

Bus 310 includes a component that permits communication among multiplecomponents of device 300. Processor 320 is implemented in hardware,firmware, and/or a combination of hardware and software. Processor 320is a central processing unit (CPU), a graphics processing unit (GPU), anaccelerated processing unit (APU), a microprocessor, a microcontroller,a digital signal processor (DSP), a field-programmable gate array(FPGA), an application-specific integrated circuit (ASIC), or anothertype of processing component. In some implementations, processor 320includes one or more processors capable of being programmed to perform afunction. Memory 330 includes a random-access memory (RAM), a read onlymemory (ROM), and/or another type of dynamic or static storage device(e.g., a flash memory, a magnetic memory, and/or an optical memory) thatstores information and/or instructions for use by processor 320.

Storage component 340 stores information and/or software related to theoperation and use of device 300. For example, storage component 340 mayinclude a hard disk (e.g., a magnetic disk, an optical disk, and/or amagneto-optic disk), a solid-state drive (SSD), a compact disc (CD), adigital versatile disc (DVD), a floppy disk, a cartridge, a magnetictape, and/or another type of non-transitory computer-readable medium,along with a corresponding drive.

Input component 350 includes a component that permits device 300 toreceive information, such as via user input (e.g., a touch screendisplay, a keyboard, a keypad, a mouse, a button, a switch, and/or amicrophone). Additionally, or alternatively, input component 350 mayinclude a component for determining location (e.g., a global positioningsystem (GPS) component) and/or a sensor (e.g., an accelerometer, agyroscope, an actuator, another type of positional or environmentalsensor, and/or the like). Output component 360 includes a component thatprovides output information from device 300 (via, e.g., a display, aspeaker, a haptic feedback component, an audio or visual indicator,and/or the like).

Communication interface 370 includes a transceiver-like component (e.g.,a transceiver, a separate receiver, a separate transmitter, and/or thelike) that enables device 300 to communicate with other devices, such asvia a wired connection, a wireless connection, or a combination of wiredand wireless connections. Communication interface 370 may permit device300 to receive information from another device and/or provideinformation to another device. For example, communication interface 370may include an Ethernet interface, an optical interface, a coaxialinterface, an infrared interface, a radio frequency (RF) interface, auniversal serial bus (USB) interface, a Wi-Fi interface, a cellularnetwork interface, and/or the like.

Device 300 may perform one or more processes described herein. Device300 may perform these processes based on processor 320 executingsoftware instructions stored by a non-transitory computer-readablemedium, such as memory 330 and/or storage component 340. As used herein,the term “computer-readable medium” refers to a non-transitory memorydevice. A memory device includes memory space within a single physicalstorage device or memory space spread across multiple physical storagedevices.

Software instructions may be read into memory 330 and/or storagecomponent 340 from another computer-readable medium or from anotherdevice via communication interface 370. When executed, softwareinstructions stored in memory 330 and/or storage component 340 may causeprocessor 320 to perform one or more processes described herein.Additionally, or alternatively, hardware circuitry may be used in placeof or in combination with software instructions to perform one or moreprocesses described herein. Thus, implementations described herein arenot limited to any specific combination of hardware circuitry andsoftware.

The number and arrangement of components shown in FIG. 3 are provided asan example. In practice, device 300 may include additional components,fewer components, different components, or differently arrangedcomponents than those shown in FIG. 3. Additionally, or alternatively, aset of components (e.g., one or more components) of device 300 mayperform one or more functions described as being performed by anotherset of components of device 300.

FIG. 4 is a flow chart of an example process 400 for modifying readableand focusable elements on a page during execution of automated scripts.In some implementations, one or more process blocks of FIG. 4 may beperformed by a client device (e.g., client device 210). In someimplementations, one or more process blocks of FIG. 4 may be performedby another device or a group of devices separate from or including theclient device, such as a web server (e.g., web server 220), a webaccessibility platform (e.g., web accessibility platform 230), and/orthe like.

As shown in FIG. 4, process 400 may include initiating an automatedscript to perform one or more interactions with a browser application(block 410). For example, the client device (e.g., using computingresource 245, processor 320, memory 330, storage component 340, inputcomponent 350, output component 360, communication interface 370, and/orthe like) may initiate an automated script to perform one or moreinteractions with a browser application, as described above.

As further shown in FIG. 4, process 400 may include identifying a set ofelements in a page rendered by the browser application that satisfy oneor more accessibility criteria, wherein the set of elements include atleast one content element that includes text readable by a screen readerapplication and at least one focusable element that is navigable using akeyboard (block 420). For example, the client device (e.g., usingcomputing resource 245, processor 320, memory 330, storage component340, input component 350, output component 360, communication interface370, and/or the like) may identify a set of elements in a page renderedby the browser application that satisfy one or more accessibilitycriteria, as described above. In some implementations, the set ofelements include at least one content element that includes textreadable by a screen reader application and at least one focusableelement that is navigable using a keyboard.

As further shown in FIG. 4, process 400 may include modifying content onthe page to make the text associated with the at least one contentelement inaccessible to the screen reader application and to make the atleast one focusable element inaccessible using the keyboard (block 430).For example, the client device (e.g., using computing resource 245,processor 320, memory 330, storage component 340, input component 350,output component 360, communication interface 370, and/or the like) maymodify content on the page to make the text associated with the at leastone content element inaccessible to the screen reader application and tomake the at least one focusable element inaccessible using the keyboard,as described above.

As further shown in FIG. 4, process 400 may include inserting at leastone explanatory element into the page based on the one or moreinteractions that the automated script is performing, wherein the atleast one explanatory element includes text that is readable by thescreen reader application to describe the one or more interactions thatthe automated script is performing, and wherein the at least oneexplanatory element is navigable using the keyboard (block 440). Forexample, the client device (e.g., using computing resource 245,processor 320, memory 330, storage component 340, input component 350,output component 360, communication interface 370, and/or the like) mayinsert at least one explanatory element into the page based on the oneor more interactions that the automated script is performing, asdescribed above. In some implementations, the at least one explanatoryelement includes text that is readable by the screen reader applicationto describe the one or more interactions that the automated script isperforming. In some implementations, the at least one explanatoryelement is navigable using the keyboard.

Process 400 may include additional implementations, such as any singleimplementation or any combination of implementations described belowand/or in connection with one or more other processes describedelsewhere herein.

In some implementations, the client device may cache an original stateof the content on the page and restore the cached original state of thecontent on the page based on determining that the automated script hasfinished executing. In some implementations, when caching the originalstate of the content on the page, the client device may store anoriginal state of the at least one content element in association with afirst unique identifier and store an original state of the at least onefocusable element in association with a second unique identifier. Insome implementations, the first unique identifier and the second uniqueidentifier may be used to restore the original state of the at least onecontent element and the original state of the at least one focusableelement when the automated script has finished executing.

In some implementations, the client device may store, in a cache, anoriginal state of the content on the page, and the original state of thecontent on the page may be removed from the cache based on determiningthat the automated script has caused the browser application to load anew page.

In some implementations, the text associated with the at least onecontent element may be defined in one or more of an attribute associatedwith the at least one content element or a section between an openingtag and a closing tag of the at least one content element.

In some implementations, when modifying the content on the page to makethe at least one focusable element inaccessible using the keyboard, theclient device may associate the at least one focusable element with anattribute that causes the at least one focusable element to be omittedfrom a sequence of elements that are navigable using the keyboard.

In some implementations, the client device may detect multimedia on thepage that is configured to automatically play, and the content on thepage may be further modified to disable the multimedia and to make themultimedia inaccessible to the screen reader application and thekeyboard.

In some implementations, the set of elements may further include atleast one interactive element. In some implementations, the content onthe page is further modified to make the at least one interactiveelement non-interactive.

Although FIG. 4 shows example blocks of process 400, in someimplementations, process 400 may include additional blocks, fewerblocks, different blocks, or differently arranged blocks than thosedepicted in FIG. 4. Additionally, or alternatively, two or more of theblocks of process 400 may be performed in parallel.

FIG. 5 is a flow chart of an example process 500 for modifying readableand focusable elements on a page during execution of automated scripts.In some implementations, one or more process blocks of FIG. 5 may beperformed by a client device (e.g., client device 210). In someimplementations, one or more process blocks of FIG. 5 may be performedby another device or a group of devices separate from or including theclient device, such as a web server (e.g., web server 220), a webaccessibility platform (e.g., web accessibility platform 230), and/orthe like.

As shown in FIG. 5, process 500 may include initiating an automatedscript to perform one or more interactions with a browser application(block 510). For example, the client device (e.g., using computingresource 245, processor 320, memory 330, storage component 340, inputcomponent 350, output component 360, communication interface 370, and/orthe like) may initiate an automated script to perform one or moreinteractions with a browser application executing, as described above.

As further shown in FIG. 5, process 500 may include identifying a firstelement in a page rendered by the browser application that satisfies oneor more accessibility criteria wherein the first element includes one ormore of text that is readable by a screen reader application or anattribute that causes the first element to be navigable using a keyboard(block 520). For example, the client device (e.g., using computingresource 245, processor 320, memory 330, storage component 340, inputcomponent 350, output component 360, communication interface 370, and/orthe like) may identify a first element in a page rendered by the browserapplication that satisfies one or more accessibility criteria, asdescribed above. In some implementations, the first element includes oneor more of text that is readable by a screen reader application or anattribute that causes the first element to be navigable using akeyboard.

As further shown in FIG. 5, process 500 may include modifying the firstelement to be inaccessible to the screen reader application and thekeyboard (block 530). For example, the client device (e.g., usingcomputing resource 245, processor 320, memory 330, storage component340, input component 350, output component 360, communication interface370, and/or the like) may modify the first element to be inaccessible tothe screen reader application and the keyboard, as described above.

As further shown in FIG. 5, process 500 may include inserting, into thepage, a second element that is readable by the screen reader applicationbased on the one or more interactions that the automated script isperforming, wherein the second element includes text to describe the oneor more interactions that the automated script is performing (block540). For example, the client device (e.g., using computing resource245, processor 320, memory 330, storage component 340, input component350, output component 360, communication interface 370, and/or the like)may insert, into the page, a second element that is readable by thescreen reader application based on the one or more interactions that theautomated script is performing, as described above. In someimplementations, the second element includes text to describe the one ormore interactions that the automated script is performing.

As further shown in FIG. 5, process 500 may include restoring the pageto an original state based on determining that the automated script hasfinished executing (block 550). For example, the client device (e.g.,using computing resource 245, processor 320, memory 330, storagecomponent 340, input component 350, output component 360, communicationinterface 370, and/or the like) may restore the page to an originalstate based on determining that the automated script has finishedexecuting, as described above.

Process 500 may include additional implementations, such as any singleimplementation or any combination of implementations described belowand/or in connection with one or more other processes describedelsewhere herein.

In some implementations, the client device may store, in one or morememories, the original state of the page prior to modifying the firstelement and inserting the second element. In some implementations, theoriginal state of the page may be obtained from the one or more memoriesand used to restore the page to the original state. In someimplementations, when storing the original state of the page, the clientdevice may associate an original state of the first element with aunique identifier and associate the first element with the uniqueidentifier after modifying the first element. In some implementations,the unique identifier may be used to restore the first element to theoriginal state of the first element when the automated script hasfinished executing.

In some implementations, the first element may be modified to include anattribute that causes the second element to be omitted from a sequenceof elements that are navigable using the keyboard. In someimplementations, the second element may be associated with an attributethat causes the second element to be first in a sequence of elementsthat are navigable using the keyboard.

In some implementations, the client device may provide informationrelated to the original state of the page and information related to oneor more modifications to the first element to another device configuredto update the one or more accessibility criteria based on theinformation related to the original state of the page and the one ormore modifications to the first element.

Although FIG. 5 shows example blocks of process 500, in someimplementations, process 500 may include additional blocks, fewerblocks, different blocks, or differently arranged blocks than thosedepicted in FIG. 5. Additionally, or alternatively, two or more of theblocks of process 500 may be performed in parallel.

FIG. 6 is a flow chart of an example process 600 for modifying readableand focusable elements on a page during execution of automated scripts.In some implementations, one or more process blocks of FIG. 6 may beperformed by a client device (e.g., client device 210). In someimplementations, one or more process blocks of FIG. 6 may be performedby another device or a group of devices separate from or including theclient device, such as a web server (e.g., web server 220), a webaccessibility platform (e.g., web accessibility platform 230), and/orthe like.

As shown in FIG. 6, process 600 may include initiating a script toautomate one or more interactions with a browser application (block610). For example, the client device (e.g., using computing resource245, processor 320, memory 330, storage component 340, input component350, output component 360, communication interface 370, and/or the like)may initiate a script to automate one or more interactions with abrowser application, as described above.

As further shown in FIG. 6, process 600 may include identifying, in apage rendered by the browser application, a set of elements that includetext readable by a screen reader application (block 620). For example,the client device (e.g., using computing resource 245, processor 320,memory 330, storage component 340, input component 350, output component360, communication interface 370, and/or the like) may identify, in apage rendered by the browser application, a set of elements that includetext readable by a screen reader application, as described above.

As further shown in FIG. 6, process 600 may include inserting at leastone explanatory element into the page based on the one or moreinteractions that the script is performing, wherein the at least oneexplanatory element includes text that is readable by the screen readerapplication to describe the one or more interactions that the script isperforming (block 630). For example, the client device (e.g., usingcomputing resource 245, processor 320, memory 330, storage component340, input component 350, output component 360, communication interface370, and/or the like) may insert at least one explanatory element intothe page based on the one or more interactions that the script isperforming, as described above. In some implementations, the at leastone explanatory element includes text that is readable by the screenreader application to describe the one or more interactions that thescript is performing.

As further shown in FIG. 6, process 600 may include modifying a documentobject model associated with the page to make the text associated withthe set of elements inaccessible to the screen reader application and tomake the text associated with the at least one explanatory elementaccessible to the screen reader application (block 640). For example,the client device (e.g., using computing resource 245, processor 320,memory 330, storage component 340, input component 350, output component360, communication interface 370, and/or the like) may modify a documentobject model associated with the page to make the text associated withthe set of elements inaccessible to the screen reader application and tomake the text associated with the at least one explanatory elementaccessible to the screen reader application, as described above.

As further shown in FIG. 6, process 600 may include providinginformation related to an original state of the document object modeland one or more modifications to the document object model to a deviceconfigured to derive one or more accessibility criteria for modifyingpage elements based on the information related to the original state ofthe document object model and the one or more modifications to thedocument object model (block 650). For example, the client device (e.g.,using computing resource 245, processor 320, memory 330, storagecomponent 340, input component 350, output component 360, communicationinterface 370, and/or the like) may provide information related to anoriginal state of the document object model and one or moremodifications to the document object model to a device configured toderive one or more accessibility criteria for modifying page elementsbased on the information related to the original state of the documentobject model and the one or more modifications to the document objectmodel, as described above.

Process 600 may include additional implementations, such as any singleimplementation or any combination of implementations described belowand/or in connection with one or more other processes describedelsewhere herein.

In some implementations, the text readable by the screen readerapplication may be defined in one or more of an attribute that describesa content element or a section between an opening tag and a closing tagof the content element.

In some implementations, the client device may associate each element inthe set of elements with an attribute having a negative value to causethe set of elements to be omitted from a sequence of elements that arenavigable using a keyboard. In some implementations, the client devicemay associate the at least one explanatory element with an attributehaving a value to cause the at least one explanatory element to be firstin a sequence of elements that are navigable using a keyboard.

In some implementations, the client device may detect multimedia thatthe page is configured to automatically play, and the client device maymake the multimedia inaccessible to the screen reader application.

In some implementations, the set of elements may further include atleast one interactive element, and the document object model associatedwith the page may be further modified to make the at least oneinteractive element non-interactive.

Although FIG. 6 shows example blocks of process 600, in someimplementations, process 600 may include additional blocks, fewerblocks, different blocks, or differently arranged blocks than thosedepicted in FIG. 6. Additionally, or alternatively, two or more of theblocks of process 600 may be performed in parallel.

The foregoing disclosure provides illustration and description, but isnot intended to be exhaustive or to limit the implementations to theprecise form disclosed. Modifications and variations may be made inlight of the above disclosure or may be acquired from practice of theimplementations.

As used herein, the term “component” is intended to be broadly construedas hardware, firmware, or a combination of hardware and software.

Certain user interfaces have been described herein and/or shown in thefigures. A user interface may include a graphical user interface, anon-graphical user interface, a text-based user interface, and/or thelike. A user interface may provide information for display. In someimplementations, a user may interact with the information, such as byproviding input via an input component of a device that provides theuser interface for display. In some implementations, a user interfacemay be configurable by a device and/or a user (e.g., a user may changethe size of the user interface, information provided via the userinterface, a position of information provided via the user interface,and/or the like). Additionally, or alternatively, a user interface maybe pre-configured to a standard configuration, a specific configurationbased on a type of device on which the user interface is displayed,and/or a set of configurations based on capabilities and/orspecifications associated with a device on which the user interface isdisplayed.

It will be apparent that systems and/or methods described herein may beimplemented in different forms of hardware, firmware, or a combinationof hardware and software. The actual specialized control hardware orsoftware code used to implement these systems and/or methods is notlimiting of the implementations. Thus, the operation and behavior of thesystems and/or methods are described herein without reference tospecific software code—it being understood that software and hardwarecan be designed to implement the systems and/or methods based on thedescription herein.

Even though particular combinations of features are recited in theclaims and/or disclosed in the specification, these combinations are notintended to limit the disclosure of various implementations. In fact,many of these features may be combined in ways not specifically recitedin the claims and/or disclosed in the specification. Although eachdependent claim listed below may directly depend on only one claim, thedisclosure of various implementations includes each dependent claim incombination with every other claim in the claim set.

No element, act, or instruction used herein should be construed ascritical or essential unless explicitly described as such. Also, as usedherein, the articles “a” and “an” are intended to include one or moreitems, and may be used interchangeably with “one or more.” Further, asused herein, the article “the” is intended to include one or more itemsreferenced in connection with the article “the” and may be usedinterchangeably with “the one or more.” Furthermore, as used herein, theterm “set” is intended to include one or more items (e.g., relateditems, unrelated items, a combination of related and unrelated items,and/or the like), and may be used interchangeably with “one or more.”Where only one item is intended, the phrase “only one” or similarlanguage is used. Also, as used herein, the terms “has,” “have,”“having,” or the like are intended to be open-ended terms. Further, thephrase “based on” is intended to mean “based, at least in part, on”unless explicitly stated otherwise. Also, as used herein, the term “or”is intended to be inclusive when used in a series and may be usedinterchangeably with “and/or,” unless explicitly stated otherwise (e.g.,if used in combination with “either” or “only one of”).

What is claimed is:
 1. A method, comprising: initiating, by a device, anautomated script to perform one or more interactions with a browserapplication executing on the device; identifying, by the device, a setof elements in a page rendered by the browser application that satisfyone or more accessibility criteria, wherein the set of elements includeat least one content element that includes text readable by a screenreader application executing on the device and at least one focusableelement that is navigable using a keyboard associated with the device;modifying, by the device, content on the page to make the textassociated with the at least one content element inaccessible to thescreen reader application and to make the at least one focusable elementinaccessible using the keyboard associated with the device; andinserting, by the device, at least one explanatory element into the pagebased on the one or more interactions that the automated script isperforming, wherein the at least one explanatory element includes textthat is readable by the screen reader application to describe the one ormore interactions that the automated script is performing, and whereinthe at least one explanatory element is navigable using the keyboardassociated with the device.
 2. The method of claim 1, furthercomprising: caching an original state of the content on the page; andrestoring the cached original state of the content on the page based ondetermining that the automated script has finished executing.
 3. Themethod of claim 2, wherein caching the original state of the content onthe page includes: storing an original state of the at least one contentelement in association with a first unique identifier; and storing anoriginal state of the at least one focusable element in association witha second unique identifier, wherein the first unique identifier and thesecond unique identifier are to be used to restore the original state ofthe at least one content element and the original state of the at leastone focusable element when the automated script has finished executing.4. The method of claim 1, further comprising: storing, in a cache, anoriginal state of the content on the page; and removing the originalstate of the content on the page from the cache based on determiningthat the automated script has caused the browser application to load anew page.
 5. The method of claim 1, wherein the text associated with theat least one content element is defined in one or more of an attributeassociated with the at least one content element or a section between anopening tag and a closing tag of the at least one content element. 6.The method of claim 1, wherein modifying the content on the page to makethe at least one focusable element inaccessible using the keyboardassociated with the device includes: associating the at least onefocusable element with an attribute that causes the at least onefocusable element to be omitted from a sequence of elements that arenavigable using the keyboard associated with the device.
 7. The methodof claim 1, further comprising: detecting multimedia on the page that isconfigured to automatically play, wherein the content on the page isfurther modified to disable the multimedia and to make the multimediainaccessible to the screen reader application and the keyboard.
 8. Themethod of claim 1, wherein: the set of elements further include at leastone interactive element, and the content on the page is further modifiedto make the at least one interactive element non-interactive.
 9. Adevice, comprising: one or more memories; and one or more processors,communicatively coupled to the one or more memories, configured to:initiate an automated script to perform one or more interactions with abrowser application executing on the device; identify a set of elementsin a page rendered by the browser application that satisfy one or moreaccessibility criteria, wherein the set of elements include at least onecontent element that includes text readable by a screen readerapplication executing on the device and at least one focusable elementthat is navigable using a keyboard associated with the device; modifycontent on the page to make the text associated with the at least onecontent element inaccessible to the screen reader application and tomake the at least one focusable element inaccessible using the keyboardassociated with the device; and insert at least one explanatory elementinto the page based on the one or more interactions that the automatedscript is performing, wherein the at least one explanatory element isnavigable using the keyboard associated with the device.
 10. The deviceof claim 9, wherein the one or more processors, when modifying thecontent on the page, are to: detect one or more interactive elements onthe page; and modify the content on the page to make the one or moreinteractive elements unselectable.
 11. The device of claim 9, whereinthe one or more processors are further configured to: store, in the oneor more memories, an original state of the page prior to: modifying theat least one content element and the at least one focusable element, andinserting the at least one explanatory element, wherein the originalstate of the page is to be obtained from the one or more memories andused to restore the page to the original state.
 12. The device of claim9, wherein the one or more processors, when modifying the content on thepage to make the text associated with the at least one content elementinaccessible to the screen reader application, are to: modify thecontent on the page by removing the at least one content element fromthe page.
 13. The device of claim 9, wherein the one or more processorsare further configured to: store, in a cache, an original state of thecontent on the page; and remove the original state of the content on thepage from the cache based on determining that the automated script hascaused the browser application to load a new page.
 14. The device ofclaim 9, wherein the one or more processors are further configured to:modify the content on the page to disable one or more multimediaelements.
 15. A non-transitory computer-readable medium storinginstructions, the instructions comprising: one or more instructionsthat, when executed by one or more processors on a device, cause the oneor more processors to: initiate an automated script to perform one ormore interactions with a browser application executing on the device;identify a set of elements in a page rendered by the browser applicationthat satisfy one or more accessibility criteria, wherein the set ofelements include at least one content element that includes textreadable by a screen reader application executing on the device and atleast one focusable element that is navigable using a keyboardassociated with the device; modify content on the page to make the textassociated with the at least one content element inaccessible to thescreen reader application and to make the at least one focusable elementinaccessible using the keyboard associated with the device; and insertat least one explanatory element into the page based on the one or moreinteractions that the automated script is performing.
 16. Thenon-transitory computer-readable medium of claim 15, wherein theexplanatory element includes one or more attributes that make theexplanatory element navigable using the keyboard.
 17. The non-transitorycomputer-readable medium of claim 15, wherein the text associated withthe at least one content element is defined in a semantic header toassist in navigation of the at least one content element.
 18. Thenon-transitory computer-readable medium of claim 15, wherein the one ormore instructions, when executed by the one or more processors, furthercause the one or more processors to: store an original state of the atleast one content element in association with a first unique identifier;and store an original state of the at least one focusable element inassociation with a second unique identifier, wherein the first uniqueidentifier and the second unique identifier are to be used to restorethe original state of the at least one content element and the originalstate of the at least one focusable element when the automated scripthas finished executing.
 19. The non-transitory computer-readable mediumof claim 15, wherein the one or more instructions, when executed by theone or more processors, further cause the one or more processors to:restore one or more original elements on the page based on determiningthat the automated script has completed executing.
 20. Thenon-transitory computer-readable medium of claim 15, wherein the one ormore instructions, when executed by the one or more processors, furthercause the one or more processors to: store, in a cache, an originalstate of the content on the page; and remove the original state of thecontent on the page from the cache based on determining that theautomated script has caused the browser application to load a new page.